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Abstract 

Being actively engaged in a task is often associated with critical thinking. Cultivating critical thinking 
skills, such as purposefully reflecting and analyzing one’s own thinking, is a major goal of higher 
education. However, there is a challenge in providing college students opportunities to clearly 
demonstrate these skills in online courses. This research explores the effectiveness of incorporating word 
clouds—visual representations of word frequency in a given passage of text—into online discussions. We 
sought to establish whether implementing word clouds in online discussions would result in a higher 
incidence of critical thinking and engagement. Survey results from undergraduate participants (n=132) 
revealed that students analyzing text in word clouds reported moderately higher scores on critical thinking 
and engagement than students analyzing the text in a linear fashion. A positive relationship was found 
between critical thinking and engagement, as well as peer interaction. This strategy can be applied to a 
wide range of educational environments to stimulate critical thinking and engagement. 


Introduction 

Supporting students to develop critical thinking skills is an overarching goal in higher education 
(Behar-Horenstein & Niu, 2011), yet it remains among the most nebulous of teaching goals in academia 
today. It has been variously defined as “the correct assessment of statement” (Ennis, 1962, p. 81), 
“reasonably reflective thinking that is focused on deciding what to believe or do” (Ennis, 1987, p. 10), 
“calling into question the assumptions underlying our customary, habitual ways of thinking and acting 
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and then being ready to think and act differently on the basis of this critical questioning” (Brookfield, 
1987, p. 1), the ability to identify, assess, and make arguments (Tsui, 2002), and finally as “not only an 
act of following the rules of logic; it is also both an act and attitude of rebellion” (Waggoner, 2014, p. 40). 
McLean (2005) notes that some definitions of critical thinking focus on both process and product, while 
others focus exclusively on one or the other. For purposes of this article, critical thinking is defined as the 
ability to purposefully reflect and challenge one’s own thinking while engaging in tasks that require the 
evaluation, analysis, and application of previous knowledge. College students with developed critical 
thinking skills have the ability to evaluate their own arguments as well as the arguments of others, resolve 
conflicts, and generate well-reasoned resolutions to complex problems (Behar-Horenstein & Niu, 2011). 
Given that there is an exponential increase in the information and knowledge being generated, it is 
extremely important that college graduates have the ability to make swift, yet well-informed decisions. 
Possessing critical thinking skills fulfills the higher education goal of nurturing students to become 
responsible citizens in a complex society. 

Engagement, also a nebulous term, is often described as another desired goal in higher education. 
In a meta-analysis of educational studies about engagement, Fredricks, Blumenfeld, and Paris (2004) 
described three types of engagement: behavioral (participating), emotional (valuing), and cognitive 
(making a concerted effort). Engaging activities tend to activate prior knowledge, foster active 
investigation, promote interaction, allow for choice, and nurture independent thinking (Jablon & 
Wilkinson, 2006). For purposes of this study, engagement is defined as the perception of enjoyment, 
interest, and motivation to complete a given academic task. Previous research has found that engagement 
is positively related to gains in critical thinking (see Tsui, 2002). In other words, as critical thinking 
increases, engagement is likely to increase as well. 

The rise of online university programs has prompted a serious conversation among educators 
about the efficacy of online teaching and learning, as there is a challenge in providing students the 
opportunities to clearly demonstrate critical thinking skills and meaningfully engage in an often 
asynchronous environment (Joyner, 2012). Asynchronous online discussions are often used to encourage 
students to think and engage with one other (Comer & Lenaghan 2012). However, mixed findings have 
been found about the effectiveness of online discussions in stimulating critical thinking. Online 
discussions rarely exhibit the highest levels of cognitive presence such as connecting ideas and 
viewpoints, and applying ideas to other contexts (Cleveland-Innes & Campbell 2012; Darabi, Arrastia, 
Nelson, Cornille, & Liang, 2011; Lee, 2014). In addition, the “limited responsiveness and interactivity 
found in many online discussions suggests a lack of attention to the ideas of others.. .many students 
interpret discussion participation as being more about ‘making posts’ than engaging in dialog” (Wise, 
Speer, Marbouti, & Hsiao, 2013, p. 324). It is necessary to conduct classroom research in order to identify 
specific online discussion strategies that have the ability to support not only a high level of critical 
thinking, but also more engaging peer interaction. 

Words clouds, which are visual representations of a document’s text (Kaptein, Hiemstra, & 
Kamps, 2010), emerge as a potential strategy to support both critical thinking and engagement within the 
context of online discussions. A word cloud takes the most frequently used words in a particular text and 
randomly displays them by size, based on their frequencies (DaPaolo & Wilkinson, 2014). Word clouds 
also display variation in color, typography, and composition, offering an aesthetically pleasing look. 
Because of this, word clouds are used in a myriad of creative and playful ways by a broad array of users 
(Viegas, Wattenberg, & Feinberg, 2009). Wordle, a website containing a tool for making word clouds, 
was launched in June 2008 and over 600,000 word clouds were created within its first nine months. 
Viegas et al. (2009) shares, “Users seem to revel in the possible applications: Scientists wordle genetic 
functions, fans wordle music videos, teacher wordle literary texts, spouses wordle love letters.... Wordles 
have graced YouTube videos, homemade gifts, posters, scrapbooks, birthday cards and valentines” 
(1137). Word clouds are also being implemented more in academic spaces. DaPaolo and Wilkinson 
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(2014) offer several ways they could be used to assess learning, such as providing a graphical 
representation of student learning, analyzing papers or writing, and comparing responses. 

With word clouds quickly coming into prominence within the academic environment in general, 
it is important to explore their potential for use in the college classroom. For educators striving to 
promote critical thinking and engagement in the online classroom, word clouds consequently possess 
potential as pedagogical tools within asynchronous discussions. Therefore, the focus of this study is to 
gauge the effectiveness of incorporating word clouds in online discussions in promoting critical thinking 
and engagement within the context of college classrooms. 

Literature Review 


Online Discussions 

Certain strategies within online discussions have been linked to fostering and promoting critical 
thinking. Szabo and Schwartz (2011) found that the use of online discussion forums as an instructional 
tool in face-to-face courses improved students’ critical thinking skills. Strategies matter, however. For 
instance, Richardson and lee (2010) found that students generally scored lower in critical thinking 
achievement levels in open-ended discussions, even though students reported preferring this format to 
others. While these results are promising, examining the effects of specific strategies in online discussions 
remains generally understudied in research. 

In an article about future directions for online discussions, Gao, Zhang, and Franklin (2013) 
recommended that future work should explore the integration of emerging technologies to address the 
constraints of current environments, the design of multifunctional environments, and the development of 
appropriate activities for these environments. Studying the impact of word clouds within online 
discussions serves to fulfill this call. 

Word Clouds in the Classroom 

Although some teachers who use word clouds in the classroom have published anecdotal 
evidence about its efficacy in promoting engagement or improving teaching strategies (see Fluisman, 
Miller, & Trinoskey, 2011; Nickel, 2012; Perry, 2012), there is growing empirical evidence regarding the 
effectiveness of using word clouds in online class settings (Bandeen & Sawin, 2012; Bromley, 2013; 
Edybum, 2010) and for qualitative assessment of student performance (DePaolo & Wilkinson, 2014). For 
example, Baralt, Pennestri, and Selvandin (2011) asked students to create word clouds from their own 
writing and found that students generated more vocabulary and new grammatical tenses, and student 
comments exhibited engagement and peer interaction. They recommended word clouds be used to 
promote reflection and brainstorming, as well as defining main ideas. Word clouds have also been used 
effectively in other areas of primary research (Aheam, 2013; McNaught & Lam, 2010). 

There is less known concerning the incorporation of word clouds into online discussions. A 
notable exception is Joyner’s (2012) study, which evaluated the use of word clouds in a discussion 
assignment. Joyner found putting students’ posts into word clouds elicited modestly greater evidence of 
critical thinking, which she identified as sharing underlying thinking, using illustrative examples, and 
connecting to a framework. In a related study, Hamm (2011) examined students as they created world 
clouds that displayed their perception of course content and posted them in an online discussion. It was 
discovered that students engaged in critical reflection of their own work and the work of classmates found 
the activity fun. Both studies had students analyze their own writing in word cloud form. Little research 
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has been uncovered that concerns word clouds containing other forms of text. 

Given that previous preliminary research suggests the association of word clouds with 
engagement and critical thinking, the purpose of the present study is to analyze the effectiveness of 
incorporating word clouds in online discussions regarding engagement as well as critical thinking. 

Methodology 

A mixed-methods approach was used for this research. A quasi-experimental design included the 
use of a classroom intervention using word clouds as part of a discussion prompt, followed by a survey 
where students assessed their own critical thinking and engagement experiences. Research was conducted 
in two sections of an undergraduate anthropology course, Language and Culture, in a large public 
university in the United States. One section of the course was taught in a blended format during the Fall 
2013 semester, and the other was fully online in Spring 2014. Both sections were taught by the same 
instructor and were identical in terms of course content and materials. The word cloud strategy was 
piloted by the instructor in the Spring and Summer 2013 semesters, with noticeable success. Once 1RB 
approval was secured to carry out this research study, the assignment prompt was designed and 
implemented for the Fall 2013 and Spring 2014 semesters. Since this is an initial research investigation of 
this strategy, the study approach described in this article is exploratory. While the results cannot generate 
strong generalizable results, they may lay the groundwork for future research. 

A sizeable portion of the course relies on graded discussions that take place in small groups of 8- 
10 students. Students are randomly assigned to these small groups at the beginning of the semester, and 
complete several graded discussions throughout the term. The word clouds studied here are implemented 
in Discussion 5 (see Appendix A), which takes place close to the end of the term, after students have had 
the opportunity to get to know one another and interact in their small groups. Fifty percent of the groups 
were randomly presented with two famous speeches in the form of word clouds (see Figure 1 and Figure 
2) and given a discussion prompt. Students in the word cloud condition were also provided a list of the 
words present in the speech, and how often they were used. The other 50% were given the same speeches 
in linear form and presented with an identical prompt (Appendix A). All students were asked to analyze 
the speeches and speculate about who might have written them, at what point in history, and with what 
purpose. They were also expected to review particular course concepts and relate them to their analyses of 
the speeches. Finally, they were asked to reply to someone in their group who had a different 
interpretation. 

All students in the course sections participated in this discussion, but in order to be included in 
this study, students had to consent to have their data included in the sample. deNoyelles, the instructional 
designer for this course, physically visited the blended class to recruit students to participate. In the fully 
online section, deNoyelles posted the study information sheet as an announcement within the class 
website and emailed the class. The Fall 2013 class had 87 students registered, of which 62 (71% of 
students enrolled) opted to participate in the study. The Spring 2014 class had 127 students enrolled, of 
which 70 participants (55% of students enrolled) participated. All participants received a 3% extra credit 
incentive for participating in the study but were also presented with an alternative extra credit activity if 
they chose not to participate. 

This study took a mixed-methods approach, in that both quantitative and qualitative data was 
collected for analysis. After completing the discussion activity, all research participants completed a 
survey (Appendix B) containing 18 closed-ended items and one open-ended item that aimed to measure 
their perceptions of critical thinking and engagement, as well as peer interaction. Survey items were 
generated from the various definitions and coding schemes employed in previous articles (see Behar- 
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Horenstein & Niu, 2011; Yang, Newby, & Bill, 2005). Cronbach’s alpha based on standardized items for 
the critical thinking scale was .70 and engagement was .84, which are acceptable levels for reliability. The 
quantitative data was analyzed using a t-test to determine if the means of the two groups (word cloud and 
linear) were significantly different concerning critical thinking. The open-ended item, which simply asked 
students to provide feedback about the discussion, was analyzed for emerging themes related to critical 
thinking and engagement and used to complement and expand understanding of the quantitative results. 
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Figure 1. First speech in word cloud form. 
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Figure 2. Second speech in word cloud form. 
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Findings 

A t-test revealed that students analyzing the text in word clouds reported higher scores on critical 
thinking than students analyzing the same text in a linear form, F { 1, 114) = 1.86, p = .176. They also had 
higher scores on engagement, F( 1, 49) = 1.60, p = .211. A positive correlation was found between 
engagement and critical thinking (r = .63, p < .05). Therefore, if a student found the discussion engaging, 
it is likely he/she also reported a higher score of critical thinking. A positive correlation was also found 
between critical thinking and peer interaction (r = .30, p < .05). Seventy-one percent of students in the 
word cloud condition agreed or somewhat agreed that classmate interaction was promoted through the 
activity, while only 52% of linear condition students said the same. 

There are a couple of explanations for these moderate, though not statistically significant, results. 
First, the sample was relatively small in size. Second, participants in both groups (word cloud and linear) 
indicated in the survey that they felt engaged in the discussion activity, in part because it was a change 
from the traditional question-and-answer discussion activities that had come before in the course. 
Students in both conditions noted the challenge of guessing the speech authors’ identities as engaging. 

While statistical inference is a valuable tool in research, it is not the sole determinant of important 
findings (Hays, 1973). The open-ended comments from the survey serve to illuminate the underlying 
effect of the word cloud on students’ critical thinking and engagement. Themes from the survey 
comments include the change in thinking, quality of learning, and peer interaction. 

Change in Thinking 

Students in the word cloud condition identified a change in thinking, which was often associated 
with a sense of engagement. For instance, one student wrote, “1 thought the assignment was definitely 
more interesting than past discussions. It made me think and analyze material from class in a new, 
interesting way.” Another explained, “1 think this discussion format was very helpful for me. It kept me 
interested and forced me to think differently than I normally would.” Analyzing the text in word clouds 
prompted the act of problem solving. Two students noted that it encouraged them to “think outside of the 
box,” which was associated with overall enjoyment of the task. This change in thinking was also 
associated with a sense of creativity. One student shared, “1 was able to use more of my creativity and 
look at things more in depth than 1 would for a regular assignment.” This stands in contrast to a student 
from the linear condition who wrote, “1 was simply analyzing a text without a name, just as 1 would 
analyze information from a textbook...1 don’t think it made me think differently than 1 normally do.” 

While encouraging students to think more freely, the word clouds subtly fostered objective 
thinking. One student shared, “The one thing 1 really liked about this assignment was how it forced me to 
think more objectively.” This was noted in another comment, “1 think that this is a valuable tool to use as 
a method of analyzing patterns in speeches from a fresh and objective perspective....” Analyzing the 
words separate from the full context of the speeches resulted in a more constructive approach. While the 
reading process was quite different in the word cloud condition compared to the linear condition, one 
student reported reading more closely, which made it a unique and enjoyable experience. A student 
reflected, “It definitely forced me to think about the speeches a little bit more. [Word clouds] force you to 
look at specific words and derive meaning from those individual terms instead of just reading an entire 
sentence and taking it for what it is.” Another student explained further, “Considering the speeches 
outside of their sentence syntax and concrete cultural context provided for an unusual framework from 
which to assess the intention and power behind the words themselves. That perspective shift made the 
assignment more enjoyable and thought-provoking, I think.” This association of engagement with critical 
thinking was often noted in the survey comments. While some in the linear condition also acknowledged 
this (“1 really enjoyed this assignment since we had to deconstruct the speeches and utilize these pieces in 
order to provide evidence for our claims”), it seems presenting the text only in word clouds may facilitate 
this process. 
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Quality of Learning 

Students in the word cloud condition often reported an enhancement in the quality of learning, 
and again related to perceptions of engagement. One student wrote, “This was the most engaged 1 have 
felt all semester. It was a more fun way to approach daunting concepts and helped to work through them 
to a more solid understanding.” Bringing in the real-world examples of the speeches helped make the 
course concepts come alive. A student explained, “1 thought this assignment was the most fun from the 
entire course so far, which is really rewarding because it is also the assignment where 1 learned the most. 
It really helped me put into practice the concepts that we are learning, which helps me understand them 
better.” By comparison, some students in the linear condition acknowledged something similar: “1 found 
the assignment more challenging when we were asked to use the textbook to back up our opinions on the 
speeches. It wasn't impossible, but took more thought processing.” However, an element of the learning 
quality only noted in the word cloud condition was the act of purposeful reflection; “Not only did it 
require us to apply concepts learned within the class, but it allowed for us to reflect on our own thought 
processes as we attempted to decode the speeches.” 

Peer Interaction 

A noteworthy difference between the word cloud and linear groups was the frequency in which 
peer interaction was mentioned in the survey. Five students in the word cloud condition specifically 
lauded the peer interaction that transpired, as opposed to one in the linear condition. They felt this activity 
“definitely ignited conversation among my peers and helped us all think more critically.” One student 
found it interesting to “see what cue words were most important for peers, while another reported, “I 
found this assignment very interesting and enlightening as to how I think compared to my peers.” This 
interaction prompted further ways of thinking about the text. A student reflected, “After I read a different 
response, it made me ponder this assignment in a completely different way...for me, it added an 
additional dimension to analyzing other’s words.” It is not surprising that positive correlation was found 
between critical thinking and peer interaction (r = .30, p < .05). Comments from four of the five students 
in the linear condition concerning peer interaction were not positive. One explained, “While some people 
will actually read through and respond thoughtfully, I feel as though more students will just type anything 
to get credit.” This echoes the sentiment of Wise et al. (2013), who reported that student participation is 
sometimes perceived to be more about fulfilling the posting requirements rather than engaging in 
dialogue. It seems the challenge of “decoding” the word cloud provided an effective talking point for 
students to thoughtfully engage in dialogue. 

Discussion and Conclusion 

Supporting the development of students’ critical thinking skills is a priority in higher education. 
With the rise in online courses, mixed results have been found regarding critical thinking within online 
discussions. This study is significant in that it provides empirical evidence suggesting that analyzing text 
in word cloud form within online discussions is associated with higher levels of critical thinking. This 
analysis prompts a change in traditional thinking, encourages a more objective consideration of the text, 
and facilitates the construction of meaning. In addition, reflection about the process of thinking is 
illuminated, within the individual as well as among peers. Engagement was also found in higher amounts 
in the word cloud condition, and a positive relationship was found between critical thinking and 
engagement. This suggests when students are critically thinking, they are more likely to find the task 
enjoyable, interesting, and motivating. With such variability, word clouds can be incorporated by many 
teachers in many ways to stimulate critical thinking (and in turn, engagement) in online discussion 
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forums, depending on the overall objectives. 

The findings implicitly suggest the conditions in which word clouds may be an especially 
effective strategy in real world use. First, it is an ideal way to encourage open exploration of a concept or 
topic without being hindered by a large amount of text. Second, it allows the instructor to gauge students’ 
prior and current knowledge, as well as overall approach to the concepts. Most importantly, analyzing text 
in word cloud forms is especially effective when the process is more important than right-or-wrong 
answers. In this study, most students guessed the authors of the speeches incorrectly, but did exhibit the 
critical thinking skills that can be generalized to other tasks. 

Based on the survey comments from students and our reflections as practitioners and researchers, 
there are some recommendations that could improve this particular assignment, as well as other 
assignments involving word clouds. For example, one student explained, “1 don’t really understand word 
clouds, so 1 didn’t really understand the assignment.” It is important to clearly articulate the nature of 
word clouds in general, and the origins of the particular word cloud, before proceeding with an 
assignment. It is also recommended educators engage students in word cloud analyses as a whole group 
prior to incorporating it as an individual assignment, as a few students reported difficulty. As one student 
shares, “It was difficult to analyze just the words without a context of sentences.” However, it should be 
noted that the analysis of word clouds is, and should be, challenging. One student explained, “The word 
cloud doesn’t really give a good context for an analysis of the speeches and makes it quite a bit harder to 
figure out what is being talked about. 1 think that the only reason 1 was able to ‘see’ what was being 
talked about was that something just clicked in my brain.” Finally, attention should be focused on the 
word cloud with regards to the location and placement of the words. If the software allows, certain words 
should be kept together in order to preserve certain phrases crucial for understanding. 

Although the present study bears promising results, it is important to note the research is limited 
in scope and should be considered exploratory. First, the sample size was small in nature. Therefore, it is 
recommended future studies incoiporate a larger sample size of participants. This study included one 
course, one teacher, one university, and one online discussion. There are nearly countless ways to 
incorporate word clouds in online discussions, and this study features a single approach. Future research 
should include involving varied types of classes and college settings, as well as trying out different 
techniques with the word clouds. For instance, students have the ability to create their own word clouds 
based on their own writings. The critical thinking that results could be very different from reading other 
people’s words. Also, it is possible that students found the discussion engaging simply because it was a 
deviation from the norm. It is recommended in future studies to employ this strategy several times to 
gauge variation in engagement and critical thinking. 

For these researchers, one thing is clear: despite the challenges of online teaching and learning, 
online discussions contain great potential for ways to creatively promote critical thinking and engagement 
in both online and blended settings, when careful consideration is given to the strategy employed. 
Purposefully selecting strategies that engage students and enhance critical thinking, such as the 
incorporation of word clouds, helps educators to cultivate these skills in college students, nurturing these 
citizens to analyze evidence and make informed decisions in a world of complex problems. 
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Appendices 


Appendix A. Discussion Assignment 5. 

We’re doing something a little different for our final graded discussion of the semester! For this 
assignment, 1 selected two speeches written at different points in history written by two people involved 
in important social movements. Fd like you to: 

Click on your group to view the speeches. They are contained in the first post. 

• FOR WEDNESDAY: Analyze these speeches, and speculate about who might have written them, 
at what point in history, and with what purpose. Explain what particular words tipped you off, 
and why. Getting the author “right” isn’t as important as analyzing the text for themes and 
meanings. Once you have analyzed the texts, compare them to each other and reflect on how the 
use of language might have changed over time. 

• FOR FRIDAY: Once you submit your answer, go back to Module 8. Review the concept of 
performance as a display or verbal artistry and perfomiativity (both in Austin's construction of 
language as action and Butler's work on gender and perfomiativity). Flow might these theories 
inform your analysis of these texts? Take some time to give this some thought. Then, click on 
"reply" which is right underneath your first post, and reflect on how these theories can inform 
your interpretation of the text above (do not edit your first post). 
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• ALSO FOR FRIDAY: Next, pick someone in your group who has a different guess or a different 
interpretation, and respond to them. 

• FINALLY: Take the Discussion Survey to provide feedback about this activity 

Do not Google the texts, it will ruin the fun! I will reveal who wrote what passage after the assignment 
has closed :) 

Appendix B. Survey Questions (critical thinking items marked with *; engagement items marked with A ) 
5-point Likert scale: Agree-somewhat agree-neither agree nor disagree-somewhat disagree-disagree 


1. The assignment instructions were clear. I understood what I was supposed to do. 

2 . A I enjoyed doing this assignment. 

3. A I was more motivated to complete this assignment than other discussions. 

4. A I found this assignment: intriguing; exciting; both intriguing and exciting; neither intriguing nor 
exciting 

5. I think this assignment is valuable. 

6 . *This assignment required me to use my critical thinking abilities. 

7. *Not knowing the names of the authors of the speeches encouraged me to think about the speech 
in a more objective manner. 

8 . *This assignment challenged the way I think. 

9. A Having to guess the authors of the speeches made the discussion activity more engaging. 

10. *This discussion activity encouraged me to think about the class content in a new way. 

11. *This assignment encouraged me to write about how I think rather than what I think. 

12. A This discussion activity held my attention longer than other discussion activities. 

13. A This assignment promoted interactions with my classmates. 

14. *Reading my peers' responses encouraged me to reflect on the way I thought about the 
discussion. 

15. *This assignment encouraged me to think "outside of the box." 

16. It took me an excessive amount of time to complete this assignment. 

17. How much time did you spend on this assignment? 

18. A This assignment should be used in future classes. 

19. Please use this space to give me any additional feedback about this assignment. 
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